Method and system for negotiation in multi-agent optimization

ABSTRACT

A method for negotiating a solution to a joint optimization problem includes: transmitting a machine-readable constraint proposal specifying one or more machine-readable constraints to each of a plurality of agents; and receiving, from each of the plurality of agents, a machine-readable counter-proposal specifying one or more additional or alternative machine-readable constraints. The method further includes computing a solution compatible with each of the one or more machine-readable constraints and the one or more additional or alternative machine-readable constraints; and communicating, to each of the plurality of agents via a machine-to-machine protocol, the computed solution.

FIELD

The present invention relates to computational optimization, and inparticular, to computational optimization in the context of negotiatingsolutions to joint optimization problems.

STATEMENT OF FUNDING

This project has received funding from the European Union's Horizon 2020research and innovation programme under grant agreement No. 769141.

BACKGROUND

In situations where multiple entities (i.e. agents) need to agree on acommon solution to an optimization problem, when a proposed solution ispresented to one such entity, said entity may wish to verify that thereis no solution in a solution space that is both better for said entitythan the proposed solution and equally fair to the other entities. Insuch situations, the entities can often (a) have different interests(objectives) and (b) have limited computational capabilities resultingin them being dependent on other entities to perform computationaloptimizations-even though they do not necessarily trust those otherentities. Such situations often arise, for example, in logisticsoperations involving multiple stakeholders (e.g. companies, drivers,customers, etc.).

Terelius (i.e. Terelius, Häkan, Ufuk Topcu, and Richard M. Murray.“Decentralized multi-agent optimization via dual decomposition.” IFACproceedings volumes 44.1 (2011): 11245-11251.), which is herebyincorporated by reference herein, describes multi-agent optimization insituations where individual agents only possess local information.

Furthermore, in Wangerman (i.e. Wangerman, John P., and Robert F.Stengel. “Optimization and coordination of multiagent systems usingprincipled negotiation.” Journal of Guidance, Control, and Dynamics 22.1(1999): 43-50.), which is hereby incorporated by reference herein, amechanism for principled negotiation between agents is proposed.However, in the mechanism described by Wangerman, negotiation is basedon the proposal and counter-proposal of full action plans, and finding asolution is likely to take many rounds negotiation. As a result, eachagent has the computationally intensive burden of computing full actionplans at each round of the negotiation in order to participate in thenegotiation.

U.S. Patent Application Publication No. 2017/0061386 describes anautomatic scheduling system that negotiates with recipients to findimportant conditions or constraints. However, the automatic schedulingsystem described therein is limited to finding suitable schedules for anumber of recipients, and the constraints described therein onlyrepresent simple availability information regarding the participants inspecific time slots. Therefore, the automatic scheduling systemdescribed therein cannot be utilized in more complex situations such asthose presented in logistics or public transport applications.Furthermore, the automatic scheduling system described therein islimited to finding a suitable schedule purely through constraintsatisfaction techniques and cannot be applied for negotiating a solutionto an optimization problem with objective functions.

SUMMARY

In an embodiment, the present invention provides a method fornegotiating a solution to a joint optimization problem, which includes:transmitting a machine-readable constraint proposal specifying one ormore machine-readable constraints to each of a plurality of agents; andreceiving, from each of the plurality of agents, a machine-readablecounter-proposal specifying one or more additional or alternativemachine-readable constraints. The method further includes computing asolution compatible with each of the one or more machine-readableconstraints and the one or more additional or alternativemachine-readable constraints; and communicating, to each of theplurality of agents via a machine-to-machine protocol, the computedsolution.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be described in even greater detail belowbased on the exemplary figures. The invention is not limited to theexemplary embodiments. All features described and/or illustrated hereincan be used alone or combined in different combinations in embodimentsof the invention. The features and advantages of various embodiments ofthe present invention will become apparent by reading the followingdetailed description with reference to the attached drawings whichillustrate the following:

FIG. 1 is a schematic diagram of an environment in which a protocol fornegotiating a solution to an optimization problem can be executed;

FIG. 2 is a flow chart illustrating a protocol for negotiating asolution to an optimization problem;

FIG. 3 is a schematic diagram of an alternative environment in which aprotocol for negotiating a solution to an optimization problem can beexecuted; and

FIG. 4 is a flow chart illustrating an alternative protocol fornegotiating a solution to an optimization problem.

DETAILED DESCRIPTION

According to embodiments of the invention, sets of messages are definedthat can be transmitted between agents to ensure fairness andverifiability of solution.

According to embodiments of the invention, multi-agent communicationmethodologies are provided that enable agents to negotiate a solution toa joint optimization problem with or without fairness constraints, andwith or without individual objective functions of the agents. In anembodiment, a central tool is to transfer, between the agents,information that provides one or more of (a) additional constraints ofthe optimization problem and (b) dual solutions as certificates ofnon-existence of certain solutions.

According to embodiments of the invention, a method is provided fornegotiating a solution to a joint optimization problem. The methodincludes transmitting a machine-readable constraint proposal specifyingone or more machine-readable constraints to each of a plurality ofagents, receiving, from each of the plurality of agents, amachine-readable counter-proposal specifying one or more additional oralternative machine-readable constraints, and computing a solutioncompatible with each of the one or more machine-readable constraints andthe one or more additional or alternative machine-readable constraints.The method also includes communicating, to each of the plurality ofagents via a machine-to-machine protocol, the computed solution.

In an embodiment the method can further include receiving, from arespective one of the plurality of agents, a respective machine-readablecounter-proposal specifying a machine-readable problematic constraint.The machine-readable problematic constraint is a constraint that willunnecessarily increase a cost of the solution to the joint optimizationproblem. The problematic constraint is a constraint that is in additionto the one or more additional or alternative constraints. The computedsolution can be a solution that does not satisfy the machine-readableproblematic constraint.

In an embodiment, the method can further include transmitting, to therespective one of the plurality of agents, a dual solution that includesthe machine-readable problematic constraint. The dual solution can, forexample, enable the respective one of the plurality of agents to verify,without computing a primal solution to the optimization problem, thatthe problematic machine-readable constraint will unnecessarily increasea cost of the solution to the joint optimization problem. The dualsolution can, for example, be a certificate of the non-existence of asolution that both satisfies the machine-readable problematic constraintand has a decreased cost relative to the computed solution. The dualsolution can, for example, define a set of functions that can be addedto the used to simplify the joint optimization problem to a simplifiedjoint optimization problem.

In an embodiment, determining a primal solution the joint optimizationproblem has a time dependence on a number of tasks to be performed thatis exponential, and determining a primal solution to the simplifiedjoint optimization problem has a time dependence on a number of tasks tobe performed that is linear.

In an embodiment, the joint optimization problem is defined by a set ofvariables, an objective function to be minimized, and a set ofconstraint predicates. The one or more machine-readable constraints ofthe machine-readable constraint proposal can be in addition toconstraints of the set of constraint predicates.

In an embodiment, the method further includes translating at least aportion of the one or more machine-readable constraints into ahuman-readable format and displaying the translated constraints on auser interface. In an embodiment, the method further includes receivinguser input via the user interface and translating the user input into atleast a portion of the one or more additional or alternativemachine-readable constraints. The at least a portion of the one or moremachine-readable constraints that is translated into a human-readableformat can be, for example, selected based on a list of translatableconstraints stored at a database.

According to embodiments of the invention, a non-transitory computerreadable medium is provided having instructions stored thereon forcarrying out a method for negotiating a solution to a joint optimizationproblem. The method includes transmitting a machine-readable constraintproposal specifying one or more machine-readable constraints to each ofa plurality of agents, receiving, from each of the plurality of agents,a machine-readable counter-proposal specifying one or more additional oralternative machine-readable constraints, and computing a solutioncompatible with each of the one or more machine-readable constraints andthe one or more additional or alternative machine-readable constraints.The method also includes communicating, to each of the plurality ofagents via a machine-to-machine protocol, the computed solution.

According to an embodiment of the invention, a coordinator agentconfigured to negotiate a solution to a joint optimization problem isprovided. The coordinator agent includes one or more processorsconfigured to transmit a machine-readable constraint proposal specifyingone or more machine-readable constraints to each of a plurality ofagents, receive, from each of the plurality of agents, amachine-readable counter-proposal specifying one or more additional oralternative machine-readable constraints, and compute a solutioncompatible with each of the one or more machine-readable constraints andthe one or more additional or alternative machine-readable constraints.The one or more processor are further configured to communicate, to eachof the plurality of agents via a machine-to-machine protocol, thecomputed solution.

In the context of this invention, a multi-agent optimization problem ischaracterized by a set of variables X, and a set of constraintpredicates dom(X)→{true, false} that have to be satisfied by anyfeasible solution. The constraints can encode technical or logicalfeasibility requirements (e.g., each parcel must be picked up by exactlyone delivery vehicle), or they can encode previously agreed fairnessconditions (e.g., no vehicle is assigned to more than twice as manyparcels compared to any other vehicle). Each agent A potentially has itsown objective function U_(A):X→IR, and we assume without loss ofgenerality here that all objectives are minimization objectives. In anembodiment, the constraint predicates are initial constraints that aredetermined prior to the coordinator agent proposing the additionalconstraints C. In an embodiment, each of the constraint predicates arein the form A*x-b≥0.

In an embodiment, the following message payloads between agents aredefined:

Constraint Proposal: an agent A proposes a set of additional constraintsC:dom(X)→{true,false} to another agent B. Here, X is the set ofvariables over which the optimization problem is defined and C is theadditional constraint proposed by the agent A. See, for example, FIG. 1.In a preferred embodiment, the constraint defines a property of thesolution that agent B is requested to agree on, e.g. a set of tasks thatB is requested to execute. The concept of constraint proposals includealso the proposal of a complete solution to the given problem (e.g. byequality constraints setting each variable from X to a specific value).

Dual Solution Transmission: an agent A transmits a dual solution toagent B. In its most general form, a dual solution consists of afunction for each constraint, where the function value is smaller thanzero if and only if the constraint is violated. In an embodiment, theconstraints are of the form Ax-b≥0, where x represents the input, A is amatrix, b is a vector, and the inequality (i.e., the above≥relationship)is component-wise. Here a dual solution can be represented as a vector uhaving the same dimensionality as b, and the corresponding constraintfunction is then defined as (Ax-b)*u, where the outer multiplication iscomponent-wise. From the dual solution, a lower bound on the value ofany (primal) solution satisfying all the constraints can be easilycomputed with respect to arbitrary objective functions. In a preferredembodiment, the dual solution having value v is used as an easilyverifiable certificate that satisfying all constraints will result in nobetter than objective value v. In an embodiment, the dual solutionhaving value v is a lowerbound value v, which is determined from solvinga simplified optimization problem derived using the dual solution u.

In the field of computational optimization, dual solutions representlower bounds on the value of an optimal primal solution. The primalsolution is a solution in the common sense, i.e. an assignment of thevariables in X to a set of values. A dual solution enables a muchsimpler optimization problem to be obtained. Specifically, a dualsolution defines a set of functions that can be added to theoptimization problem's objective function after all constraints havebeen removed thereby providing a simpler optimization problem. The bestsolution to the resulting simplified optimization problem represents alower bound on the best solution to the original optimization problem.More specifically, the best solution which satisfies all constraints(that are covered by the dual solution) cannot have a value better thansaid lower bound.

Because the simplified optimization problem can be solved with littlecomputational effort (e.g. in the case of linear optimization problems,solving the optimization problem exhibits linear time-dependence),agents that do not have extensive computational power can compute thelower bound from a given dual solution. As a result, they do not need totrust the correctness of the result presented by another agent.

According to an embodiment of the invention, a negotiation protocol isprovided for determining a solution to a multi-agent optimizationproblem. In a first step, a specific agent, acting as the coordinator,sends a constraint proposal C_(A) to each of the other agents A involvedin the optimization (encoding, for example, a set of tasks among whicheach agent is requested to execute a subset).

In a second step, each of the other agents either rejects the proposalor returns one or more further constraints to be satisfied. Each of theother agents A rejects the proposal if it will not accept any solutioncompatible with the proposed constraint proposal C_(A) (meaning, forexample, that it will not execute any of the requested tasks).Alternatively, each of the other agents A sends back another constraintproposal C_(A)′ if a solution compatible with constraint proposal C_(A)will be accepted if the solution additionally satisfies the constraintproposal C_(A)′ (for example, the agent is willing to execute at most 3of the proposed tasks).

In a third step, the coordinator, having received the responses,declares the negotiation process to have failed if some agent hasrejected the proposal. Otherwise, it computes an optimal solutionsatisfying all C_(A) and C_(A)′. The solution it is then presented toall agents by a third constraint proposal. In situations where one ofthe agents has rejected the proposal in the second step or no feasiblesolution satisfying all C_(A) and C_(A)′ is found in the third step, thecoordinator can either declare the negotiation process as failed, or canstart again at the first step by providing a different constraintproposal C_(A) to the agents.

According to a second embodiment of the invention, after an agreedcommon objective function (e.g. to minimize the total cost) has beenidentified, a coordinator can collect constraint proposals from allagents (as in the second step of the previous embodiment). However, whensome agent A proposes a constraint C whose satisfaction will lead to ahigh overall cost and there is an alternative solution with a much lowercost, then the coordinator can return the following information to A:(i) a cheaper alternative solution with value v′ that violates C and/or(ii) a dual solution to the original problem augmented with the proposedconstraint C, whose value v is larger than the value v′ of thealternative solution. This pair of solutions certifies that thesatisfying the proposed constraint C will unnecessarily increase theoverall costs. Because the values of both primal and dual solutions areeasy to compute, the agent A can efficiently verify the correctnesswithout the need (a) to trust the correctness of the coordinator'scomputations and (b) to compute any primal or dual solution by itself.In an embodiment, a commonly agreed objective function is a prerequisiteof the dual solutions process. In an embodiment, transmitting constraintproposals back and forth remains possible without a commonly agreedobjective function.

According to a third embodiment of the invention, human feedback can beincorporated into the negotiation process. For example, humans (e.g.drivers of trucks) can be involved in a negotiation process wherebysolution to an optimization problem is determined. As humans are,without extensive training, not able to process and understand genericconstraints of optimization processes, the constraints need to betranslated by a user interface and presented in a way that the human canmake a decision about it or make a counterproposal. For example, theconstraint proposal might be presented as a list of tasks, from whichthe human can select some subset or reject all of them. Because not allconstraints can be translated in such a way, the coordinator agent inthis embodiment has some database of translatable constraints, and itwill only propose such types of constraints to the agent user interface.

According to embodiments, solutions are determined to logisticsoptimization problems are in which several participating serviceproviders need to agree on a coordinated schedule for pickup anddelivery of goods. In such cases, constraints represent proposals ofwhich actor performs which tasks, and dual solutions representcertificates that prior proposals will lead to high costs. For example,a logistics optimization problem could be to minimize delivery costsubject to the following constraints: (a) each parcel gets picked up anddelivered, and (b) each vehicle is on the road for no more than 8 hours.In the negotiation process, the coordinator agent presents to eachvehicle a set of parcels to pick up and deliver in the form of aconstraint. The vehicles can return, as a counter-proposal, a constraintwhich encodes a smaller subset of the proposed parcels to pickup anddeliver.

According to embodiments of the invention, solutions are determined tologistics optimization problems for scenarios where multiple deliveryservice providers share some infrastructure and services. Such scenariosare not uncommon for logistics/postal companies-even when they arecompeting for customers in the same market segment and area. In somemarkets (e.g. postal services in some cities), sharing of infrastructureand services is even mandated by regulation. According to suchembodiments, ad-hoc negotiations as to how to service upcoming pickupand delivery orders can be performed in a time-efficient manner. Herethe common objective function can represent regulatory or contractuallyagreed upon key performance indicators (KPIs), e.g. an amount ofemissions produced, or a total number of kilometers traveled.Constraints sent between the agents can be, e.g., price offers. Forexample, such a logistics optimization problem could be to minimize atotal route traveled, subject to the constraint that all parcels aredelivered without delay, i.e. within a specified period of time orbefore a specified delivery date and/or time. In identifying a solutionto such an optimization problem, a first delivery company, or agent, Acan make a proposal to a second delivery company, or agent, B, using aconstraint that company A will deliver a subset of the parcels andcompany B will deliver the remaining parcels. Company B can make aconstraint counter-proposal, demanding a compensation payment fordelivering the parcels as proposed by company A.

According to further embodiments of the invention, a solution isdetermined to a public transport optimization problem in which multipleagents, e.g. a control center, on-demand buses, and taxis, negotiate todetermine the specific aspects of a public transport service that eachwill provide. In said embodiments, constraints can be provided in theform of working times, kilometer limitations, and numbers of passengersto transport. Dual solutions can be used to show that satisfyingconstraints will lead to a larger amount of energy usage and longertransportation routes. For example, such a public transport optimizationproblem could be to minimize total CO2 emissions subject to thefollowing constraints: (a) all passengers are transported to theirdemanded destination, and (b) none of the vehicles exceeds a specifiedpassenger limit. A control center, acting as a coordinator agent, couldpropose to a specific vehicle to serve a specific set of customers. Thevehicle might make a counter-proposal which consists of a constraintwhich limits its total kilometers to travel.

FIG. 1 is a schematic diagram of a system environment in which aprotocol for negotiating a solution to an optimization problem can beexecuted. The system environment includes a coordinator agent 110 and aplurality of additional agents 120. The coordinator agent 110 and eachof the plurality of additional agents 120 can be, e.g., one or morephysical machines each including compute resources, storage resources,and network resources—or a component of such one or more physicalmachines. For example, each of the coordinator agent 110 and theplurality of additional agents 120 can be, e.g., a virtual machine (VM)or collection of VMs hosted by one or more physical machines. Thecompute resources of the physical machines include one or moreprocessors each having one or more processor cores. The storageresources can include computer readable memory, e.g.,random-access-memory (RAM) and/or cache memory. The network resourcescan include, e.g., a physical network interface controller. Thecoordinator agent 110 and each of the plurality of additional agents 120can include, e.g., a set of processor executable instructions stored ata processor readable memory, e.g. a processor readable memory of the oneor more physical machines, and configured to be executed by a processor,e.g. a processor or processor core of the one or more physical machines.The set of processor executable instructions can include, e.g.,instructions for carrying out a method for negotiating a solution to ajoint optimization problem.

In the system environment of FIG. 1, the coordinator agent 110 canexecute a protocol for negotiating a solution to a joint optimizationproblem. The protocol involves sending, by the coordinator agent 110, aset of proposed constraints (i.e. constraint proposal C_(A)), andreceiving, by the plurality of additional agents 120, the set ofproposed constraints. The protocol further involves sending, by each ofthe plurality of additional agents 120, a set of counter-proposalconstraints (i.e. constraint counter-proposal C_(A)′) that areadditional constraints or constraints that are alternatives to one ofmore of the constraints in the set of proposed constraints. Thecoordinator agent 110 then receives each respective set ofcounter-proposal constraints sent by each respective one of theplurality of additional agents 120. Thereafter, the coordinator agent110 determines a solution to the joint optimization problem thatsatisfies all constraints in the constraint proposal C_(A) and in eachrespective constraint counter-proposal C_(A)′. Finally, the coordinatoragent 110 communicates, or transmits, the determined solution to each ofthe plurality of additional agents 120.

FIG. 2 is a flow chart illustrating a protocol for negotiating asolution to an optimization problem. The protocol illustrated in FIG. 2can be executed by the coordinator agent 110 of FIG. 1. At 210, acoordinator agent (e.g. the coordinator agent 110) sends a constraintproposal C_(A) to each of a plurality of additional agents (e.g. theplurality of additional agents 120) involved in the optimization. Theconstraint proposal C_(A) encodes, for example, a set of tasks amongwhich each agent is requested to execute a subset.

At 220, the coordinator agent receives, from each of the plurality ofadditional agents, a response to the constraint proposal C_(A) eitherrejecting the constraint proposal C_(A) or returning one or more furtherconstraints to be satisfied, i.e. by providing a set of counter-proposalconstraints (i.e. constraint counter-proposal C_(A)′) that areadditional constraints or constraints that are alternatives to one ofmore of the constraints in the set of proposed constraints. Eachrespective one of the plurality of additional agents will reject theconstraint proposal C_(A) if it will not accept any solution compatiblewith constraint proposal C_(A) (meaning, for example, that it will notexecute any of the requested tasks). Alternatively, each respective oneof the plurality of additional agents will return a constraintcounter-proposal C_(A)′ if a solution compatible with constraintproposal C_(A) will be accepted if the solution additionally satisfiesthe additional or alternative constraints specified by the constraintcounter-proposal C_(A)′ (for example, the agent is willing to execute atmost 3 of the proposed tasks).

At 230, the coordinator agent, having received the responses from eachof the plurality of additional agents, determines whether any respectiveone of the plurality of additional agents has rejected the constraintproposal C_(A). If one or more of the plurality of additional agents hasrejected the constraint proposal C_(A), the protocol returns to 210where the coordinator agent proposes a new constraint proposal C_(A) toeach of the plurality of additional agents. Alternatively, if none ofthe plurality of additional agents has rejected the constraint proposal,the protocol proceeds to 240 where the coordinator agent determines anoptimal solution satisfying all constraints of both C_(A) and C_(A)′.Thereafter, at 250, the coordinator agent transmits, or communicates,the solution to each of the plurality of additional agents.

FIG. 3 is a schematic diagram of an alternative environment in which aprotocol for negotiating a solution to an optimization problem can beexecuted. In the alternative environment illustrated in FIG. 3, an agentuser interface 310 is provided that enables a user 320 to communicatewith the coordinator agent 110. In the environment illustrated in FIG.3, the coordinator agent 110 includes a database that stores a list oftranslatable constraints and is configured to executecomputer-executable instructions for determining which of a plurality ofconstraints of an optimization problem can be presented in a formatunderstandable to a human user and for presenting such constraints insuch format. In this manner, the coordinator agent 110 can presentcertain constraints to the human user in a manner that enables the humanuser to accept those constraints or make a counterproposal via the agentuser interface 310.

FIG. 4 is a flow chart illustrating an alternative protocol fornegotiating a solution to an optimization problem. The protocolillustrated in FIG. 4 can be executed by the coordinator agent 110 ofFIG. 1. At 410, a coordinator agent (e.g. the coordinator agent 110)sends a constraint proposal C_(A) to each of a plurality of additionalagents (e.g. the plurality of additional agents 120) involved in theoptimization. The constraint proposal C_(A) encodes, for example, a setof tasks among which each agent is requested to execute a subset. Insome embodiments, the additional agents may already possess a definitionof the multi-agent optimization problem that is to be solved. In otherwords, prior to receiving the constraint proposal C_(A) from thecoordinator agent, the additional agents may already possess adefinition of a set of variables X and a set of constraint predicatesdom(X)→{true, false}(which must be satisfied by any feasible solution)that collectively characterize the multi-agent optimization problem. Insuch embodiments, the constraints in the constraint proposal C_(A) sentby the coordinator agent represent constraints proposed by thecoordinator agent that are in addition to those in the set of constraintpredicates dom(X)→{true, false} that characterize the multi-agentoptimization problem. In such cases, the constraint predicatesdom(X)→{true, false} that characterize the multi-agent optimizationproblem can represent technical or logical feasibility requirementsand/or legal requirements, while the constraints in the constraintproposal C_(A) sent by the coordinator agent can represent parameters ofa possible solution (e.g. restrictions on the form of the solution)requested or demanded by the coordinator agent. Alternatively, theconstraint proposal C_(A) sent by the coordinator agent at 410 mayinclude a definition of a set of variables X, the constraint predicatesdom(X)→{true, false} that characterize the multi-agent optimizationproblem, and additional constraints requested or demanded by thecoordinator agent.

At 420, the coordinator agent receives, from each of the plurality ofadditional agents A, a response to the constraint proposal C_(A)returning one or more further constraints to be satisfied, i.e. byproviding a set of counter-proposal constraints (i.e. constraintcounter-proposal C_(A)′) that are additional constraints or constraintsthat are alternatives to one of more of the constraints in the set ofproposed constraints C_(A). The constraints in a respective constraintcounter-proposal C_(A)′ represent parameters of a possible solution(e.g. restrictions on the form of the solution) requested or demanded bya respective additional agent. In other words, a respective constraintcounter-proposal C_(A)′ sent by a respective additional agent indicatesthat the respective additional agent will accept a solution compatiblewith constraint proposal C_(A) if the solution additionally satisfiesthe additional or alternative constraints specified by the constraintcounter-proposal C_(A)′. However, the constraints in a respectiveconstraint counter-proposal C_(A)′ cannot be alternatives to constraintpredicates dom(X)→{true, false} that characterize the multi-agentoptimization problem.

At 430, the coordinator agent determines that one or more additional oralternative constraints C* specified in one or more respectiveconstraint counter-proposals C_(A)′ are problematic constraints, i.e.constraints that will have a detrimental impact on the desiredoptimization, e.g. unnecessarily lead to an increased value of aquantity to be minimized (such as total cost or total time tocompletion). At 440, the coordinator agent then determines, for each ofthe one or more constraints C*, (i) a primal solution (with one or morevalues v′ that violates a respective one of the one or more constraintsC*) that represents an improved, or cheaper, alternative solution to anysolution that might satisfy the respective one of the one or moreconstraints C*, and (ii) a dual solution to the original optimizationproblem (characterized by the set of variables X and the constraintpredicates dom(X)→{true, false}), where the dual solution satisfies therespective one of the one or more proposed constraints C*.

At 450, the coordinator agent transmits, to each respective additionalagent that requested each respective additional or alternativeconstraints C*, a solution pair including (i) the primal solution withthe one or more values v′ that violate the respective constraint C*, and(ii) the dual solution to the original optimization problem(characterized by the set of variables X and the constraint predicatedom(X)→{true, false}), where the dual solution satisfies the respectiveconstraint C*. The solution pair enables an additional agent to quicklyand easily—and without expending significant computationalresources-verify correctness of the coordinator agent's computations(i.e. that the primal solution with the one or more values v′ providesan optimization value, e.g. total cost, total emissions, that issuperior to the best optimization value that could be achieved by asolution that satisfies the respective constraint C*) without the needto compute any primal or dual solution itself. At 450, the coordinatoragents also transmits, to each additional agent that did not request anyof the one or more constraints C*, the primal solution with the one ormore values v′.

One or more embodiments of the invention provide advantages over priorart methods for solving multi-agent optimization problems by (i)enabling negotiation by transmitting constraint proposals, and (ii)enabling negotiation by transmitting dual solutions as certificates ofoptimality or suboptimality.

The manner in which transmission of dual solutions as certificates ofoptimality or suboptimality is illustrated by the following example.Consider, for example, a problem where a set of n tasks needs to beexecuted, and there two agents, agent 1 and 2. When agent 1 executestask i, nonnegative costs c_i1 have to be paid. The costs of agent 2executing the task are defined as c_i2 accordingly. In an embodiment,the objective function is to minimize the total cost:

minimize x_11*c_11+ . . . +x_n1*c_n1+x_12*c_12+ . . . +x_n2*c_n2

where x_i1 is an indicator variable which represents the decisionwhether agent 1 will execute task 1 (x_i1=1) or not (x_i1=0);accordingly for agent 2.

As all tasks need to be executed by at least one agent, there is aconstraint:

x_i1+x_i2≥1

for each task i=1 . . . n.

Furthermore, executing tasks requires time. Agent 1 executing task irequires time t_i1; accordingly for agent 2. Each agent has some totaltime budget available, respectively denoted T_1 and T_2. The constraintnot to exceed the time budget can be formulated by the two constraints:

x_11*t_11+ . . . +x_n1*t_n1≤T_1x_12*t_12+ . . . +x_n2*t_n2≤T_2.

As this problem is a generalization of the well-known KNAPSACK problem,determining an optimal solution with today's hard- and software takestime exponential in the number of tasks.

Suppose that agent 1 proposes to solve either task 1 or task 2, or both.This proposal is expressed by the additional constraint

x_11+x_21≥1.

Assume that both the cost c_11 and the cost c_21 are very large comparedto other costs, where c_21 is even larger than c_11. Agent 2, in orderto prove to Agent 1 that the proposal will lead to a large cost,presents to Agent 1 the following dual solution:

c_11*(1−x_11−x_21) for the additional constraint; and

0 for all other constraints.

For the additional constraint, the function satisfies the requirement ofdual solutions that the function value is 0 or lower in case theconstraint is satisfied. For the other constraints, this requirement istrivially satisfied.

Applying the dual solution, the problem is now to minimize:

x_11*c_11+ . . . +x_n1*c_n1+x_12*c_12+ . . .+x_n2*c_n2+c_11*(1−x_11−x_21)

without any constraints to satisfy.

The new objective function is linear, with the coefficient of x_11 being0, the coefficient of x_21 being (c_21-c_11>0), the other coefficientsbeing the same as in the original objective function, and the constantterm being c_11. As all coefficients are nonnegative, the minimumsolution is obtained by setting all x-variables to zero; thus theminimum value is c_11, the cost of agent 1 executing task 1.

Computing the coefficients of the new problem and verifying that allcoefficients are nonnegative can be done in linear time; thus the lowerbound c_11 can be verified by agent 1 in linear time given the dualsolution obtained from agent 2.

Embodiments of the invention provide, inter alia, the followingadvantages over prior art methods for solving multi-agent optimizationproblems:

-   -   Ability to systematically negotiate solutions to multi-agent        optimization problems by proposing constraints is more efficient        than directly negotiating solutions because less communication        rounds are required. Furthermore, communicating constraints        still leaves the freedom to select any solution the satisfies        the constraints, without being limited to a single solution that        has been proposed.    -   Verifying optimality and suboptimality via dual certificates is        computationally much more efficient than verifying this through        re-solving, by each individual agent, the whole optimization        problem.    -   Agents do not need to reveal their individual objectives and        system state; instead they can express their conditions for        cooperation in the form of constraints.

While the invention has been illustrated and described in detail in thedrawings and foregoing description, such illustration and descriptionare to be considered illustrative or exemplary and not restrictive. Itwill be understood that changes and modifications may be made by thoseof ordinary skill within the scope of the following claims. Inparticular, the present invention covers further embodiments with anycombination of features from different embodiments described above andbelow. Additionally, statements made herein characterizing the inventionrefer to an embodiment of the invention and not necessarily allembodiments.

The terms used in the claims should be construed to have the broadestreasonable interpretation consistent with the foregoing description. Forexample, the use of the article “a” or “the” in introducing an elementshould not be interpreted as being exclusive of a plurality of elements.Likewise, the recitation of “or” should be interpreted as beinginclusive, such that the recitation of “A or B” is not exclusive of “Aand B,” unless it is clear from the context or the foregoing descriptionthat only one of A and B is intended. Further, the recitation of “atleast one of A, B and C” should be interpreted as one or more of a groupof elements consisting of A, B and C, and should not be interpreted asrequiring at least one of each of the listed elements A, B and C,regardless of whether A, B and C are related as categories or otherwise.Moreover, the recitation of “A, B and/or C” or “at least one of A, B orC” should be interpreted as including any singular entity from thelisted elements, e.g., A, any subset from the listed elements, e.g., Aand B, or the entire list of elements A, B and C.

What is claimed is:
 1. A method for negotiating a solution to a jointoptimization problem, the method comprising: transmitting amachine-readable constraint proposal specifying one or moremachine-readable constraints to each of a plurality of agents;receiving, from each of the plurality of agents, a machine-readablecounter-proposal specifying one or more additional or alternativemachine-readable constraints; computing a solution compatible with eachof the one or more machine-readable constraints and the one or moreadditional or alternative machine-readable constraints; andcommunicating, to each of the plurality of agents via amachine-to-machine protocol, the computed solution.
 2. The method ofclaim 1, further comprising: receiving, from a respective one of theplurality of agents, a respective machine-readable counter-proposalspecifying a machine-readable problematic constraint, wherein themachine-readable problematic constraint is a constraint that willunnecessarily increase a cost of the solution to the joint optimizationproblem, and wherein the problematic constraint is a constraint that isin addition to the one or more additional or alternative constraints. 3.The method of claim 2, wherein the computed solution does not satisfythe machine-readable problematic constraint.
 4. The method of claim 3,further comprising transmitting, to the respective one of the pluralityof agents, a dual solution that includes the machine-readableproblematic constraint.
 5. The method of claim 4, wherein the dualsolution enables the respective one of the plurality of agents toverify, without computing a primal solution to the optimization problem,that the problematic machine-readable constraint will unnecessarilyincrease a cost of the solution to the joint optimization problem. 6.The method of claim 5, wherein the dual solution is a certificate of thenon-existence of a solution that both satisfies the machine-readableproblematic constraint and has a decreased cost relative to the computedsolution.
 7. The method of claim 4, wherein the dual solution defines aset of functions that can be added to the used to simplify the jointoptimization problem to a simplified joint optimization problem.
 8. Themethod of claim 7, wherein determining a primal solution the jointoptimization problem has a time dependence on a number of tasks to beperformed that is exponential, and wherein determining a primal solutionto the simplified joint optimization problem has a time dependence on anumber of tasks to be performed that is linear.
 9. The method of claim1, wherein the joint optimization problem is defined by a set ofvariables, an objective function to be minimized, and a set ofconstraint predicates.
 10. The method of claim 9, wherein the one ormore machine-readable constraints of the machine-readable constraintproposal are in addition to constraints of the set of constraintpredicates.
 11. The method of claim 1, further comprising translating atleast a portion of the one or more machine-readable constraints into ahuman-readable format and displaying the translated constraints on auser interface.
 12. The method of claim 11, further comprising receivinguser input via the user interface and translating the user input into atleast a portion of the one or more additional or alternativemachine-readable constraints.
 13. The method of claim 11, wherein the atleast a portion of the one or more machine-readable constraints that istranslated into a human-readable format is selected based on a list oftranslatable constraints stored at a database.
 14. A non-transitorycomputer readable medium having instructions stored thereon for carryingout a method for negotiating a solution to a joint optimization problem,the method comprising: transmitting a machine-readable constraintproposal specifying one or more machine-readable constraints to each ofa plurality of agents; receiving, from each of the plurality of agents,a machine-readable counter-proposal specifying one or more additional oralternative machine-readable constraints; computing a solutioncompatible with each of the one or more machine-readable constraints andthe one or more additional or alternative machine-readable constraints;and communicating, to each of the plurality of agents via amachine-to-machine protocol, the computed solution.
 15. A coordinatoragent configured to negotiate a solution to a joint optimizationproblem, the coordinator agent comprising: one or more processorsconfigured to: transmit a machine-readable constraint proposalspecifying one or more machine-readable constraints to each of aplurality of agents; receive, from each of the plurality of agents, amachine-readable counter-proposal specifying one or more additional oralternative machine-readable constraints; compute a solution compatiblewith each of the one or more machine-readable constraints and the one ormore additional or alternative machine-readable constraints; andcommunicate, to each of the plurality of agents via a machine-to-machineprotocol, the computed solution.